Traditional feature encoding scheme (e.g., Fisher vector) with localdescriptors (e.g., SIFT) and recent convolutional neural networks (CNNs) aretwo classes of successful methods for image recognition. In this paper, wepropose a hybrid representation, which leverages the discriminative capacity ofCNNs and the simplicity of descriptor encoding schema for image recognition,with a focus on scene recognition. To this end, we make three maincontributions from the following aspects. First, we propose a patch-level andend-to-end architecture to model the appearance of local patches, called {\emPatchNet}. PatchNet is essentially a customized network trained in a weaklysupervised manner, which uses the image-level supervision to guide thepatch-level feature extraction. Second, we present a hybrid visualrepresentation, called {\em VSAD}, by utilizing the robust featurerepresentations of PatchNet to describe local patches and exploiting thesemantic probabilities of PatchNet to aggregate these local patches into aglobal representation. Third, based on the proposed VSAD representation, wepropose a new state-of-the-art scene recognition approach, which achieves anexcellent performance on two standard benchmarks: MIT Indoor67 (86.2\%) andSUN397 (73.0\%).
展开▼
机译：具有局部描述符（例如SIFT）和最近的卷积神经网络（CNN）的传统特征编码方案（例如Fisher向量）是成功的两类图像识别方法。在本文中，我们提出了一种混合表示，它利用了CNN的判别能力和描述符编码模式的简单性来进行图像识别，重点是场景识别。为此，我们将从以下几个方面做出三个主要贡献。首先，我们提出了一个补丁程序级别的端到端架构，以对本地补丁程序的外观进行建模，称为{\ emPatchNet}。 PatchNet本质上是经过弱监督训练的定制网络，它使用图像级监督来指导补丁级特征提取。其次，我们通过利用PatchNet的强大特征表示来描述本地补丁并利用PatchNet的这些语义概率将这些本地补丁聚合为全局表示，从而提供了一种混合视觉表示，称为{\ em VSAD}。第三，基于提出的VSAD表示，我们提出了一种新的最新场景识别方法，该方法在MIT Indoor67（86.2 \％）和SUN397（73.0 \％）两个标准基准上均具有出色的性能。
展开▼
